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Cost and Schedule Analytical Techniques 

Development 

Contract NAS8-40431 


Option Year 3 Final Report 


Period Covered December 1, 1997 through November 30, 1998 

I. INTRODUCTION 

This Final Report summarizes the activities performed by Science Applications 
International Corporation (SAIC) under contract NAS 8-40431 "Cost and Schedule 
Analytical Techniques Development Contract" (CSATD) during Option Year 3 
(December 1, 1997 through November 30, 1998). This Final Report is in compliance with 
Paragraph 5 of Section F of the contract. 

This CSATD contract provides technical products and deliverables in the form of 
parametric models, databases, methodologies, studies, and analyses to the NASA 
Marshall Space Flight Center's (MSFC) Engineering Cost Office (PP03) and the 
Program Plans and Requirements Office (PP02) and other user organizations. Detailed 
Monthly Reports were submitted to MSFC in accordance with the contract’s Statement of 
Work, Section IV “Reporting and Documentation”. These reports spelled out each 
month's specific work performed, deliverables submitted, major meetings conducted, and 
other pertinent information. Therefore, this Final Report will summarize these activities 
at a higher level. 

During this contract Option Year, SAIC expended 25,745 hours in the performance of 
tasks called out in the Statement of Work. This represents approximately 14 full-time 
EPs. Included are the Huntsville-based team, plus SAIC specialists in San Diego, Ames 
Research Center, Tampa, and Colorado Springs performing specific tasks for which they 
are uniquely qualified. 


II. MSFC TASKS 

The basic CSATD contract calls out three major Statement of Work task areas that 
provide analytical techniques development activities for MSFC. Deliverables to NASA 
resulting under the basic tasks include an update version of the restricted NAFCOM96 to 
some 100 government users, delivery of an update version of unrestricted NAFCOM96 to 
several hundred users, three quarterly NAFCOM Newsletter to all NAFCOM users, and 
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several hundred users, three quarterly NAFCOM Newsletter to all NAFCOM users, and 
documented results of dozens of ad hoc, quick turn-around taskings. In addition, the 
REDSTAR library was increased by 1,148 documents, 14 additional REDSTAR CD- 
ROM disk were developed, a web server procured to establish a web-based REDSTAR, 5 
training courses were conducted to train NAFCOM users, major analytical and 
programming work was done to improve and expand the next NAFCOM release 
(NAFCOM99), and the Complexity Generator methodology was established. 
Accomplishments under these and other areas are discussed in the following paragraphs. 

II. 1. Task 1 - REDSTAR Data Base System Maintenance and Expansion 

II. 1. 1. REDSTAR Library 

Approximately 1,148 documents were added to REDSTAR during 1998, bringing the 
total number of documents in REDSTAR to 21,187 documents. REDSTAR’ s growth was 
mainly due to the continued cataloging of documents received during 1997, along with 
the acquisition of four boxes of data from MSFC PP03. Further enhancement of the 
database was achieved by acquiring a copy of Space 2000 ; a computerized historical 
database of 100,000 records containing space related activity from 1926 to 2014. To 
insure organizational stability, and to make room for future growth, a total shift and 
reorganization of the REDSTAR library took place during July and August. 

During Option Year 3, outside requests for REDSTAR documents and research assistance 
came from the following companies and NASA centers: Ball Aerospace, National Air 
Intelligence Center of Wright Patterson AFB, TRW, Lockheed Martin, NASA Earth 
Science Enterprise, JSC, LeRC, and Raytheon. Several of the requests were made as a 
result of locating the REDSTAR homepage on the Internet. 

Data collection contacts were made during the year to answer specific inquiries, and to 
add data to the REDSTAR collection. These requests included the following contacts: 
called TRW to request back issues of TRW’s Spacelog ; called Thiokol to determine the 
costs of the Star-48, Star-27, and Star-20 motors; called Pratt & Whitney to determine the 
cost of the RL10 IIB expansion bell; called the Office of NASA Chief Financial Officer 
to get REDSTAR on the distribution list for the annual updates of the NASA New Start 
inflation index; contacted MSFC historian for assistance in locating information on “man 
in the can”; and called North Carolina Foam Industries concerning external tank 
insulation. The following web-based databases were used to locate information relating to 
NASA: NASA-Recon, Carl Uncover, EBSCO, AIAA, Redstone Scientific Information 
Center, UAH Library, MSFC Technical Reports Server, MSFC Repository, JPL Mission 
and Spacecraft Library, NODIS (NASA on-line directives). Foreign & U.S. Launch 
Vehicle Log, Air University Library and Database, NASA Technical Report Server, 
NASA Astrophysics Library, General Accounting Office Reports, and X.500 NASA 
Directory Service. 
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Search request topics in 1998 included the following topics: Celestri, Polar Ultraviolet 
Imager, “man in the can”, 1990-95 global launch vehicle rates and revenues, “Cosmos on 
a Shoestring”, Mars missions schedules, commercial-off-the-shelf cost models, active 
GEO satellites, space travel and tourism, LEO commercial launch vehicle projections, aft 
cargo carrier, utilization of external tank for space station, cost information on solar 
electric propulsion, articles dealing with “faster, cheaper, better”, UHF Follow-on, Eagle 
class vehicles, EOS, RD-0120 engine cost, and FAST. 

II.l. 2. Web-based REDSTAR Library & CD-ROMs 

Fourteen additional CD’s were completed for the third REDSTAR on CD-ROM effort, 
bringing the total number of scanned REDSTAR documents to 7,500. This third set of 
2,506 documents totaled 229,218 pages. These documents were accessible from the CD- 
ROM using a searchable data base that includes all the REDSTAR cataloging 
information. In this year, we have begun to develop a Web-based version of REDSTAR 
that will provide a much wider access to the REDSTAR documents and provide better 
security of the information. 

To accommodate the web-based REDSTAR, we have procured and configured a Web 
Server. This server will also contain other cost related web pages such as the Space 
Systems Cost Analysis Group (SSCAG) homepage and a NAFCOM web page. The 
server will be located in San Diego until the REDSTAR page is operational, then it will 
be located in Huntsville. The characteristics of the server are provided below. 

♦ Compaq Proliant 6000 Server 

♦ Dual 200 megahertz Pentium Pro Processors 

♦ 256 megabytes RAM 

♦ 4-18 gigabyte RAID Level 5 drives, 1 hot spare 18 gigabyte drive 

♦ 1 00 megabit ethemet line 

♦ Uninteruptable power supply 

♦ Windows NT 4.0 operating system 

We are currently developing the user interface for the web-based version of the data base. 
This version will include password protection, searching for documents; printing 
documents or search results, downloading documents, and email access to the REDSTAR 
librarian for specialized searches or scanning of new documents. 

II. 1. 3. Data Analysis 

The NAFCOM database continues to undergo changes to incorporate more useful data for 
NASA cost estimators. During the year we have finished the first round of intense data 
collection for technical information on all NAFCOM subsystems and components. This 
data is being placed in memo fields located in the NAFCOM database. It can be accessed 
through the estimating process when one needs additional information on aerospace 
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hardware. The data, in paragraph form, will also appear in the spacecraft resumes under 
the Help option in NAFCOM. 

The addition of new data points in the 1999 NAFCOM release includes a number of 
previously collected launch vehicles. These data at the stage level include: Delta II Stage 

1, Delta II Stage 2, Delta II Stage 3, Minuteman III Stage 1, Minuteman III Stage 2, 
Minuteman III Stage 3, Minuteman III Stage 4, Peacekeeper Stage 1, Peacekeeper Stage 

2, Peacekeeper Stage 3, Peacekeeper Stage 4, AFSLV Stage 1, AFSLV Stage 2, AFSLV 
Stage 3, AFSLV Stage 4, Atlas II Stage 1, Atlas II Stage 2, Atlas II Stage 3, Titan IV 
Stage 1, Titan IV Stage 2, and Titan IV Stage 3. The addition of these data makes the 
NAFCOM launch vehicle data base an extremely powerful estimating tool. 

We continue to assist MSFC with timely response to ad hoc assignments. In the past year 
we have responded to many cost and technical questions concerning new and historical 
data. We have responded to questions and provided data on the X-33, STAR motors, data 
concerning current and past launch vehicles, “low cost” spacecraft, the AXAF spacecraft, 
studies such as “man-in-the-can”, older spacecraft such as HEAO, HST, and GRO. 
Additional data has been supplied on cost and weights for trunnion and keel fittings, 
ground-base telescopes. Space Station propulsion, and information on yearly launches, 
both in the United States and worldwide. 

II. 2. Task 2 - Development of Cost Estimating Techniques 

II. 2. 1. NAFCOM 

Primary NAFCOM-related activities performed this contract year included a database 
update for NAFCOM96, significant expansion of the NAFCOM user community, and 
continued development of NAFCOM99. 

II. 2. 1. 1. Database Update forNAFCOM96 

This year we completed and distributed an update to the NAFCOM96 model to correct a 
potential problem discovered in the system integration database average files, and to add 
four additional unmanned spacecraft data points to the database. This update was 
distributed to all Government Only and Unrestricted Release users. 

II. 2.1.2. Expansion of the NAFCOM User Community 

This contract year, SAIC has distributed 26 additional copies of the NAFCOM96 
Government Only version and 60 copies of the Unrestricted Release version. Those 
requesting and receiving copies of the Government Only version of the software 
represented organizations in the Air Force, NASA, BMDO, DARPA, and Navy. The 
Unrestricted Release version of NAFCOM was distributed to 7 foreign organizations and 
at least 33 different US contractors, individuals, and/or universities. 
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II. 2. 1. 3. Continued Development ofNAFCOM99 

NAFCOM99, due to be released early in the next contract year, will be a major update to 
the current version of NAFCOM. The next version will offer several new features along 
with some enhanced features and of course additional data points. The most notable 
features that will be introduced in NAFCOM99 are the Complexity Generator, the 
Process Based Estimating feature, the Liquid Rocket Engine Module, and the Scientific 
Instruments Estimating Module. The Complexity Generator and the Process Based 
Estimating features are discussed in II. 2. 5. and II. 3. 2. 

II. 2. 1.4. Scientific Instrument Module 

The Scientific Instrument Module provides the capability to estimate instruments’ cost 
using the traditional NAFCOM methodology and a database of over 350 instruments. The 
majority of the instruments database was obtained from GSFC’s Scientific Instruments 
Cost Model, with additional data being retrieved from the REDSTAR database. SAIC 
worked closely with the MSFC Engineering Cost Office, to develop the database and 
model interface for the scientific instruments module. Originally, the database was 
segregated by instrument class into nineteen data files. To facilitate easier filtering and 
database creation, all instrument classes were combined into one database file and 
additional fields were created for filtering data, including components fields for 
mechanisms, optics, detectors, and miscellaneous components. 

To add an instrument system the NAFCOM user will first make that selection on the Add 
WBS Element screen, as shown in Figure 1. The user will then add instruments to the 
system using the user define, specific analogy, and/or database average options familiar 
to NAFCOM users. The specific analogy database includes 89 fields of information, 28 
of which are used for filtering. In addition to the Scientific Instruments Class filter 
shown in Figure 2, users will have the option of filtering on technical characteristics such 
as size and measurement range, and programmatic characteristics such as contractor and 
contract start date. 
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Figure 1, Add WBS Element Screen 
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Figure 2, Scientific Instruments Class Filter 
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n. 2. 1.5. Liquid Rocket Engine Module 

The Liquid Rocket Engine Module in NAFCOM99 provides the capability to estimate the 
cost of liquid rocket engines using the estimating methodology set forth in Rocketdyne’s 
Liquid Rocket Engine Cost Model prepared for MSFC. 

The Liquid Rocket Engine Cost Model provides historically-based, parametric CERs at 
the engine level for estimating development and production costs of chemical propulsion, 
liquid propellant rocket engines in the 20 Klbs to 2000 Klbs vacuum thrust range. CERs 
are also provided at the component level for an alternative method of estimating the 
production costs. The cost model’s CERs give cost as a function of size, complexity and 
process attributes. The model is an engineering model and is not based on regression 
analysis, since only a few data points were used to develop the model. 

To add an engine system the NAFCOM user will first make that selection on the Add 
WBS Element screen shown previously in Figure 1. NAFCOM then allows the user to 
enter development cost inputs, and choose the detailed, component level model, or the 
summary, engine level model for estimating the production costs, as shown in Figure 3. 
Figure 4 provides the input parameters for the development and production CERs. 


Add Liquid Rocket Engine System 


WBS Item Name: 


Engine Globals 


Development Rate per Year . 

Production Rate per Year 

Production Quantity 

Fee Percentage 


30 


30 


Production: 


Ducts 


C Detailed Production Model 


Gimbals 


Thrust Chamber 


T urbo Pumps 


Cancel 


Development 

I Hardware Development | Design Engineering Labor) 
! Test Labor | Toofng.GSE, &STE | 


C Summary Production Model 


Injectors 


Main Valves 


Preburner/Gas Generator 


Help 


OK 


] 


Figure 3, Add Liquid Rocket Engine System Screen 
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Model Inputs 

Development 

hngine Cycle/lntemal hnvironment Complexity 1 est frequency 

Engineering Design/Mfg Maturity Tooling Availability 

Development Process Improvements Unit Production Cost 

Development Engine Fab. Time Span Production Rate 

Test Reduction Factor 

tngine Level 
Production 

Producibility Improvements Chamber Pressure 

Manufacturing Automation Thrust Level 

Thermodynamic Cycle Production Quantity 

Reusability Production Rate 

Propellant Type 

Component Level 
Production 

Level ot Manufacturing Support Weight 

Degree of Outsourcing Quantity Per Engine 

Component Type Production Quantity 

Number of Turbopump Parts Production Rate 


Figure 4, Liquid Rocket Engine Module Input Parameters 


In preparation for adding this feature to NAFCOM99, we reviewed the Liquid Rocket 
Engine Cost Model, seeking guidance from part of the Liquid Rocket Engine Model's 
development team at Boeing. The algorithms and appropriate input screens have been 
incorporated into NAFCOM. The Engine System status display, additions to printouts, 
and help text is currently being worked. 

II. 2. 1.6. Other Features 

In addition to these features, NAFCOM99 will include a notes/summary field, quicker 
direct access to the database, a database memo field, improved project resumes, 30 
additional projects, metrics conversion for weight, and others. The notes/summary field, 
which has been added to the main screen, continuously shows updates to totals as 
elements are added or edited, and with a click to the right mouse button also allows entry 
and display of notes pertaining to the individual elements. 

Previously, to access the NAFCOM database, the user was required to first initiate the 
process of adding a specific analogy element. Now, to simply view the database, the user 
only has to click a button in the toolbar for immediate access. 

Much effort was expended this year to complete the research for technical and 
programmatic data for the new memo fields and to improve the project resumes. In a 
NAFCOM99 database, the user can select a data point, click a button, and view a 
technical narrative about the particular subsystem or component. In addition, the 
technical and programmatic data will be incorporated into improved project resumes 
located in the help system. 

The NAFCOM database currently contains 104 projects. Data has been collected and 
analyzed for 30 new projects that will be included in NAFCOM99. 
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To satisfy numerous requests by NAFCOM users, NAFCOM99 will also accommodate 
the use of either kilograms or pounds as the unit of measure for weight. 
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Figure 5, Sample of Data Input Sheet 


In this contract year, the NAFCOM development team continued efforts to make the 
NAFCOM software development process more efficient. Early in the year, we created a 
resource-loaded network for the NAFCOM project, which uses identified tasks required 
to complete the project and the time required performing these tasks to generate a 
schedule. An accompanying Project WBS and WBS Dictionary were also created. In 
addition, we conducted 36 internal NAFCOM Planning, Definition and Review Meetings 
with NAFCOM team members. These meetings were held to track the progress of 
NAFCOM related tasks, and to make decisions concerning new and existing features. 


II. 2. 2. NAFCOM Programming 

Option Year 3 produced sweeping improvements within the NAFCOM99 model. The 
following paragraphs will briefly delineate the programming improvements made in order 
by module. 


11 




II. 2. 2. 1. Summary/Note Screen 


The most readily apparent change for NAFCOM users will be the transformation of the 
main screen. The main screen was divided to include a summary/note screen combination 
in the upper right portion of the main screen display. The default setting is a summary of 
DDT&E, Flight Unit, Production, and Total values for the Grand Total, System, 
Hardware Total, and System Integration fields. If the Summary is clicked once with the 
right mouse button, it transforms into a note field. When the note field is double clicked, 
a dialog box appears to enable the user to input unique notes on any WBS element. These 
notes will appear within the correlating area on the NAFCOM printouts. 

II. 2. 2. 2. Scientific Instruments Module 

We developed a routine for adding a Scientific Instrument data point. The process 
required for adding an Instrument was different than adding any other data point because 
System Integration was already included in Instruments. This requires writing special 
calculation procedures and changing our rule base for handling data points in an estimate. 
The calculation code was very complex, making the Add Instrument routine more 
difficult to program than a regular WBS hardware element. The Scientific Instruments 
also has to be put in a special place in the estimate so the user can easily identify the 
instrument as having no System Integration calculated in. We successfully completed all 
pre-test coding of the Scientific Instruments Module and fully integrated the module into 
NAFCOM99. 

II. 2. 2. 3. Liquid Rocket Engine Cost Model 

We began the process of incorporating the Liquid Rocket Engine Cost Model into 
NAFCOM99. This involved the creation of numerous screens and extensive coding. The 
Liquid Rocket Engine Cost Model consists of 13 separate dialog boxes. There are four 
separate Development options and two separate Production models within the Liquid 
Rocket Engine Cost Model. The analysts have completed developing the algorithms for 
the model and the coding is 50% complete. 

II. 2. 2. 4. Complexity Generator 

We received the first series of data on the Complexity Generator. We began the process 
of programming the Complexity Generator by modifying the NAFCOM99 screens to 
ready them for the integration of the Complexity Generator. We developed the Electrical 
Power and Distribution Group Subsystem screen. As the Complexity Generator 
encompasses a considerable amount of information, a tab property has been integrated 
into the dialog boxes. This allows for the user to remain on one static screen and move 
back and forth between information. The coding of this module will continue as the 
analysts provide data. 
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II. 2. 2. 5. Context Sensitive Help 


We developed context sensitive help for the NAFCOM99 application. This involved 
physically adding the help button to each dialog box within NAFCOM99. We then 
acquired the help index number for each dialog box and programmed the context 
sensitive help accordingly. 

II. 2. 2. 6. Cost Sheet Viewer 

The Cost Sheet Viewer was improved in several areas. White space was reduced within 
the Cost Sheet Viewer. We increased the size of the display screen to provide a larger 
display area. Also, to improve the viewing capability, selected columns have been frozen 
within the Cost Sheet Viewer. 

II. 2. 2. 7. FBS Matrix Update 

We updated the FBS matrix. This involved updating all of the NASA and OSD 
percentages. Along with updating the FBS Matrix, several default values in NAFCOM99 
had to be updated. 

II. 2. 2. 8. FBS Equation Update 

We reviewed the new FBS equation with the analysts. After we studied the present 
coding and naming convention of the equation, we restructured the FBS equation for 
NAFCOM99. 

II. 2. 2. 9. Inflation Indices Update 

We updated the inflation indices within NAFCOM to reflect 1998 values. We verified 
that the inflation indices were operating correctly and they were pulling the correct 
values. 

II. 2. 2. 10. NAFCOM99 Y2K Compliance 

We tested NAFCOM99 for Y2K compliance. As we had predicted before the test 
procedure, NAFCOM99 was fully Y2K compliant. 

II. 2. 2. 11. Copy and Paste Option 

We have developed a copy and paste option that will allow the user to paste data points 
between separate running copies of NAFCOM. This will allow the user to copy a data 
point from one estimate to another. In the past, we have only been able to copy a data 
point and paste it in the same estimate. 
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II. 2. 3. NAFCOM Newsletter 


In this contract year, three issues of the NAFCOM Newsletter were published and 
distributed to 362 NAFCOM users and others. The winter issue of the NAFCOM 
Newsletter introduced a NAFCOM Data Input Sheet which is now included in all 
NAFCOM distribution and training packages. The Input Sheet, as shown in Figure 1, 
assists users in collecting and organizing input parameters necessary for performing and 
documenting a NAFCOM estimate. This Input Sheet was developed to satisfy numerous 
requests made during NAFCOM training sessions. 

The Spring 1998 issue featured an in-depth article about “Process Based Estimating 
Within the NASA/ Air Force Cost Model.” The article included excerpts from a paper 
that SAIC presented at the 1998 Joint Annual Conference of the International Society of 
Parametric Analysts (IS PA) and the Society of Cost Estimating and Analysis (SCEA). 

The Fall 1998 NAFCOM Newsletter included a very in-depth discussion of NAFCOM’ s 
code complexity and Y2K compliance, as well as training updates and model tips and 
tricks. Starting with the Fall 1998 issue, the Newsletter exhibited a new, sleeker color 
format, as exhibited in Figure 6. 
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Figure 6, Fall 1998 Issue of the NAFCOM Newsletter 
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II. 2. 4. NAFCOM Training 


Five NAFCOM96 training sessions were held during this contract year, consisting of 
three Government Only classes and two Unrestricted Release classes. Estimators from 
AFCAA and Photon Research, who supports BMDO, attended the Government Only 
classes. The Unrestricted Release training sessions included analysts from Lockheed 
Martin (New Orleans) and Qualis Corporation. 

In an attempt to boost class attendance this year we mailed registration forms to one 
individual at each organization that has more than one registered NAFCOM user, in 
addition to all users that expressed interest in training or called with comments or 
questions. We also distributed registration forms at one of the monthly local SCEA 
meetings. We were contacted by Northrop Grumman, NASA Ames Research Center, 
Boeing (Huntsville, AL and Huntington Beach, CA), and Lockheed Martin (Huntsville) 
regarding formal training courses, but no classes have been scheduled as of yet. 

II. 2. 5. Complexity Generators 

The Complexity Generators represent an innovative cost estimating methodology where 
multiple cost driving variables are identified and algorithms are developed that allow 
using the cost drivers to determine the costs of future programs. We are finalizing the 
development of the NAFCOM Complexity Generators by refining our development 
approach using the electrical power subsystem as a prototype for the other subsystems. 

II. 2 5. 1. Approach 

The approach for developing the Complexity Generators includes the step identified 
below. Each of the steps will be briefly described. 

♦ Define potential cost drivers 

♦ Develop data base of cost drivers 

♦ Analyze data for trends and impacts 

♦ Categorize complexity 

♦ Develop complexity relationships 

♦ Test and verify 

♦ Documentation and demonstration 

The potential cost drivers for the Complexity Generators have evolved over the past year. 
There are different cost drivers for each of the subsystems although some cost drivers are 
the same for all. A list of candidate cost drivers for each subsystem is shown below. 
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Figure 7, Potential Cost Drivers 


II. 2. 5. 2. Database 

A data base of the cost driver information for each historical project is required to 
develop the complexity relationships. This data base of technical, programmatic, and cost 
data has been updated this year to include new cost drivers and new projects. There was 
also a considerable effort to validate and supplement the data already in the cost driver 
data base. Where actual historical data could not be located; we estimated required data 
using engineering judgement, analogy, or other estimating approaches. The data base 
consists of spreadsheets for each subsystem by mission class (e.g. Manned, Unmanned 
Earth Orbiting, Unmanned Planetary, or Launch Vehicle) with all potential cost drivers 
identified. An example of a portion of one of the spreadsheets is shown below. 
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Figure 8, Cost Driver Data Base 

After collecting the cost driver data base, we analyzed individual cost drivers for their 
impact to the cost to an electrical power subsystem. To do this we sorted the data, 
filtered on specific drivers, and normalized cost for several combinations of parameters. 
For example, we considered peak amp-hours, average amp-hours, and amp-hours per 
pound versus development cost, unit cost, development a-value, unit a-value, cost per 
pound, and technology readiness levels. The graphs shown below represent the types of 
trends we found. 
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Figure 9, Data Trends 


n.2. 5. 3. Cost Drivers 

After analysis of the cost trends for the electrical power subsystem cost drivers, we chose 
the cost drivers to use and grouped the drivers into three main categories. The grouping of 
the cost drivers into categories allows us to apply engineering judgement to the 
application of the individual cost drivers. For example, if we strictly applied statistical 
measures to determine the coefficients for the complexity equations and used each cost 
driver as a dependent variable, then some of the cost drivers would have an inverse effect 
on the cost (as the driver increased in complexity the cost would go down). The 
categories of complexity are shown below for the electrical power subsystem. 

• Inheritance 

• Technical Complexity 

— Output Power 
— Storage Capacity 
— Year of Technology 
— Power Regulation 

— Design Life 

• Project Management 

— Manufacturing Management 
— Funding Availability 
— Risk Management 
— Integration Complexity 
— Design Management 
— Pre-phase C/D Studies 
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Using the data from analysis of cost drivers for trends, we weighted each of the cost 
drivers within the three categories in relation to one another. For example, Year of 
Technology has a much larger impact on the cost than does the Design Life. For 
Inheritance, Technical and Project Management, we developed a complexity scale from 0 
to 100. Where 0 represents lowest complexity and 100 represents highest complexity. 
Using the cost drivers data base we determined the complexity for each of the complexity 
variables. We then used multiple least squares regression to determine the coefficients for 
the complexity equation. 

We developed a prototype of the Complexity Generator screen for the electrical power 
subsystem to demonstrate its use. This prototype screen was presented at the NASA Cost 
Estimating Symposium and to the Air Force Cost Analysis Agency. The screen allow the 
user to change one or more cost drivers and to see the impact on the cost estimate. The 
prototype screen is shown below. 


Complexity Generate* 
Electrical Power Subsystem Prototype 
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Integration Complexity | Moderate Major interfaces Involving Multiple Contractors/Centers "*] 

Pre-Phase C/D Study [ T wo or More Study Contracts in Phase A & B Between 9 and 1 8 Months * \ 


Management Effectiveness 


■> 


Technical Management 
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Moderately Experienced Management Team Using Dedbated Design Team Dependent on Some 
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Requirements Changes 


DtrD Cost ($M) 


Fight Unit Cost(JM) 


Technology Readiness Level (TRL) 
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Cancel 


Help 


OK 


Figure 10, Prototype Complexity Generator Screen 


We are currently refining the complexity relationships and testing the output of the cost 
estimating equations. 
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II. 3. Task 3 - Development of Schedules, Plans and Requirements 

In Option Year 3, SAIC provided schedule analysis for several projects in Program 
Development and began the development of a subsystem schedule estimating model 
based on developmental processes. These activities are summarized in the following 
paragraphs. 

II. 3. 1. General Scheduling 

Schedule analysis and development were performed on a number of projects including 
included Space Based Laser (SBL), Material Science Research Rack (MSRR), ASTOR, 
and AIRSEDS. These are short-term, quick turnaround tasks to support Quarterly 
Reviews and other briefings with updates and further schedule details. Updates for these 
projects were required every few months and all were completed in a timely manner. 

II. 3. 2. Process-Based Subsystem Schedule Estimating 

During this past year SAIC has continued efforts to develop a schedule estimating model 
based on subsystem development processes. After research of REDSTAR schedule data, 
design handbooks, guidelines, and technical papers, a preliminary schedule at the process 
level was developed for each major subsystem (i.e. structure, thermal, avionics, and 
propulsion). The processes and their relative schedule duration were reviewed with* 
discipline engineers in the Preliminary Design Office to ensure accuracy and 
completeness. A detailed set of schedule drivers was also identified for each subsystem. 
The drivers were categorized into one of four major headings: 1) Management Efficiency 
2) Requirements Changes 3) Design Inheritance and 4) Technical Complexity. The list of 
schedule drivers was then used to develop a preliminary model input form. User 
selections on the input form will be used to calculate individual process duration, thus 
determining overall subsystem schedule. Future efforts will focus on detailed algorithm 
development and programming. The schedule estimating model will be incorporated as a 
module in the NASA Air Force Cost Model (NAFCOM). 


III. OTHER TASKINGS 

In addition to the work performed directly for MSFC, synergistic tasks were performed 
for other NASA elements and the Air Force Cost Analysis Agency. These tasks were 
funded by organizations other than the MSFC Program Development. Some of these were 
direct extensions or expansions of basic contract efforts. Others made use of the 
REDSTAR library, the NAFCOM model and database, and analytical techniques 
developed under the basic contract and therefore could be performed cost effectively and 
in much short time than if they had been done under separate contracts. 

Those tasks funded by the NASA Comptroller were the continued development of the 
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Space Operations Cost Model, the calibration of PRICE Systems data for NASA users, 
partial funding of a SAIC cost analyst at the Ames Research Center, development of 
REDSTAR CD-ROMs and web-based REDSTAR, and augmentation to NAFCOM 
development. 

The MSFC Science and Engineering Directorate continued the SAIC assessment and 
validation of the COMPRE model this contract year. Ames Research Center funded the 
remaining portion of the SAIC cost analyst at Ames. The MSFC Microgravity Office 
continued this year to fund enhancements and updates to the Microgravity Experiment 
Cost Model that we had developed for them earlier. 

The Air Force Cost Analysis Agency (AFCAA) provide funding and taskings in several 
areas. We provided an independent assessment of the EELV Cost Analysis Requirement 
Document, served as a team member on the Air Force EELV Investment/Financial 
Analysis Team, and made a number of Air Force desired enhancements to NAFCOM. 

These other taskings are described in the following paragraphs. 

III. 1. Space Operations Cost Model (SOCM) 

The NASA Space Operations Cost Model (SOCM) development study is an ambitious 
task to develop a NASA agency-wide operations cost model. Operations types include 
robotic planetary and Earth orbiting science missions, as well as transportation systems, 
space facility, and ground facility operations support for robotic and/or human spaceflight 
missions. At the end of 1997, prototype models were operational for robotic space 
science missions and plans were underway for formulation of the other modules. In this 
contract year, these prototypes have been iteratively refined based on input from a 
growing user community and have gained acceptance in the costing community. Also, 
several prototypes for modeling launch systems and orbiting space facilities have been 
developed and are leading to innovative approaches for evaluating operational 
requirements of large NASA projects. 

III. 1. 1. Summary of Activities 

As the Operations Cost Model Steering Committee progresses in understanding key cost 
and technology issues associated with launch systems and the ISS, our membership has 
more than doubled. Added members include experts from KSC for launch operations, 
LaRC for STS and aircraft reliability and maintainability, MSFC for launch vehicle 
design, economics, and marketability, and JPL for ISS logistics and support assessments. 
The current Operations Cost Model Steering Committee members are shown below. Del 
Wilson from NASA HQ is the Chairman and Mark Jacobs from SAIC is his deputy. 
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Figure 11, NASA Operations Cost Model Steering Committee Members 

(Decemberl998) 


As our team develops approaches to capture operations costs for near-term RLV/ELV 
systems and the ISS, we identify ways these approaches can be used as the basis for 
evaluating mid- and far-term concepts including lunar and Mars bases and interplanetary 
transportation systems. Plans for modeling Human Spaceflight (HSF) projects are under 
review and include developing a system interface that can access each SOCM module and 
HSF-customized submodules. These plans will be evolved over the next study year. 

The new SOCM Module Family is shown below. There are four modules currently in 
rapid prototype development mode. The first two. Robotic Earth Orbiting and Planetary, 
share a common methodology with mission-unique input sets. The Launch Systems 
Module is being integrated at SAIC using inputs from MSFC, KSC, and LaRC. SAIC is 
also integrating the Space Facilities Module using inputs from JPL ISS modeling and 
data collection efforts. The Human Spaceflight Module, covering lunar/Mars exploration 
missions, will be developed from derivatives of the other four modules. 
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- Scenario Builder - combines 
estimates from other SOCM 
submodules 
* HSF submodules 

Figure 12, SOCM Module Family 

Near-term plans include (1) completion of a Version 1.0 for the Earth Orbiting and 
Planetary Modules with integrated help menus and documentation, (2) integration of the 
LaRC Reliability and Maintainability Analysis Tool (with help from Old Dominion 
University) and the KSC Vision Spaceport Study into the Launch Systems Module 
prototype, and (3) completion of the ISS operations prototype by JPL. After these 
prototypes become operational, modeling efforts will focus more on HSF lunar/Mars 
missions. 

This annual report summarizes Steering Committee activities; rapid prototype model 
(RPM) development effort status/descriptions for each module; study products, 
presentations, applications; and future plans. 

III. 1. 2. SOCM Steering Committee Activities 

The SOCM Steering Committee has added new members to support development of the 
Launch Systems and Space Facilities modules. These projects have a complex set of 
interacting cost drivers that have various impacts on life cycle costs, economics, and 
marketability. Our team is focusing on providing the necessary outputs characterizing 
operational support requirements to facilitate overall system life cycle analyses. A key 
objective for modeling operations is to capture the effects of advanced technology 
insertion in the flight and ground systems on support costs (post-development). As we 
generate prototype modules for advanced system concepts, the Steering Committee and 
SOCM user community continue to provide valuable feedback and direction to ensure 
reasonable accuracy for technology forecasts and cost impacts. 

Four Steering Committee Meetings were held over the past year. The first was at NASA 
JSC in January and included a 1-day SOCM training session in one of JSC’s training 
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facilities. At the NASA HQ/GSFC March meeting, our team was given tours of the 
operations centers for HST and TRACE. JPL provided a tour of the PDC including a 
SOCM demonstration at the June meeting. In August, the team met at NASA LaRC and 
focused on the prototyping activities supporting the Launch Systems (RLV) and Space 
Facilities (ISS) modules. 

Also over the past year, several other operations centers have been visited including Mars 
Pathfinder, Mars Surveyor, and the New Millennium Deep Space One facilities at JPL. 
These tours provide valuable insight into the working level concerns and issues that drive 
ground system requirements, designs, and advanced technology insertion. 

HI. 1. 3. RPM Status/Descriptions for Specific SOCM Modules 

SOCM currently includes five modules that have some estimating methodology 
similarities, but each with a unique set of cost drivers and estimating categories. 
Prototypes exist for four of the modules (HSF mission prototyping is planned for 1999) 
and a final Version 1 .0 of the Earth Orbiting and Planetary Modules should be complete 
early next year. 

Data collection efforts have continued and are distributed among almost all study team 
members. Technical, programmatic, and cost information is collected for recent, near- 
term, and future mission concepts from the mission set shown here. The model 
development effort combines constructive and CER-based approaches and has the 
flexibility to utilize whatever information is available. The model’s performance is 
continually enhanced as more information is collected improving our team’s 
understanding of key issues. The prototyping activities focus on recent and near-term 
mission requirements and attempt to capture the differences for past and future projects 
by using a cost driver input set that compares various operations support characteristics to 
present-day State-of-the-Practice (SoP) technology levels. 
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NASA Center 

Conventional 
Approach 
(past projects) 

Use of Low Cost Modern 
Business Practices 
(current projects/SOP) 

Future Missions 
(future projects/SO A) 

Goddard 
Space Flight 
Center 
(GSFC) 

Gamma Ray Observatory 
(GRO) 

Hubble Space Telescope 
(HST) 

Energetic UV Explorer 
(EUVE) 

Advanced Composition 
Explorer (ACE) 

Far UV Spectroscopic 
Explorer (FUSE) 

Solar, Anomalous and 
Magnetospheric Particle 
Explorer (SAMPEX) 

X-Ray Timing Explorer (XTE) 

Midex (MAP, IMAGE) 
SMEX (TRACE, FAST, 
SWAS. WIRE) 

ESSP 

EOS 

NMP EO Missions 

Marshall Space 
Flight Center 
(MSFC) 


Advanced X-ray Astrophysics 
Facility (AXAF) 

Space Station 

Mars Exploration 
Advanced Launch Vehicles 

Johnson Space 
Center (JSC) 

Shuttle Orbiter 

Space Station 

Mars Exploration 

Jet Propulsion 

Laboratory 

(JPL) 

Galileo 

Magellan 

Voyager 

Discovery Program (Mars 
Pathfinder, NEAR/APL) 
Mars Global Surveyor (MGS) 

New Millennium DS Missions 
Discovery Program (Lunar 
Prospector, Stardust) 

Pluto Flyby 

Kennedy Space 
Center (KSC) 

Shuttle Orbiter 


HRST. RLV 


Figure 13, SOCM Reference Mission Set 


Capturing advanced technology impacts is a key objective for the modeling effort to 
improve the cost community’s understanding of life cycle cost (LCC) trades from 
investments in flight and ground hardware and operations systems. As the SoP advances 
inserting improved technologies into new missions, LCC can be reduced for missions 
with similar levels of performance to historical analogies. However, if advanced 
technology is applied to maximize performance, LCC will eventually increase. The 
following figure graphically characterizes this relationship. Inputs for the SOCM modules 
compare a concept’s requirements and advanced technology implementation to SoP 
values to develop relative differences that can be correlated to cost differences (that can 
either increase or decrease operations costs). 

SOCM module status and description summaries are provided in the following 
paragraphs. 


Planetary and Earth Orbiting Modules . Several iterations have been completed with 
these modules incorporating feedback from a growing user community and enhancing the 
capabilities and performance with each RPM version. Enhancements include: 

Revised estimating methodology using multiple Level 1 FTE ranges separating 
engineering, navigation, and science operations 

New Earth orbiting science drivers based on inputs from NASA GSFC Code 600/900 
scientists 

Incorporation of SOMO operations services costs 

File integration feature enabling combinations of multiple SOCM runs 

Separate versions for Office ’95 and ’97 on PC and Macintosh systems 


25 



Life 

Cycle 

Cost 


Improved 
Technology/ 
Performance 
drives the LCC 
vs. Performance 
relationship 


Less data 
High margins 


LCC reduction eventually 
reduces Performance 


1990 State-of-the-Practice 


More data Current State-of-the-Practice 

More resources / 
for payloads 

Performance increases 
eventually drive LCC up 


1 - Baseline Technology 

2 - Adv Tech used to reduce cost 

3 - Adv Tech used to increase performance! 



Performance 


Figure 14, Capturing Advanced Technology Application 

A key feature of the Planetary and Earth Orbiting modules is the ability to generate 
multiple output formats, as shown on the following page. These output options include 13 
“traditional” functions that correlate to past project data collected, an activity-based 
output separating activity requirements for each major project element, and a high-level 
summary separating flight/engineering, navigation, and science support. Providing 
multiple result formats facilitates flexibility for unique model applications such as 
determining specific impacts of adding payload items to a mission or estimating post- 
flight data analysis staffing/cost requirements. 

Launch Systems Module . The Launch Systems RPM has progressed significantly over the 
past year. The current approach estimates staffing and costs for 12 launch operations 
support functions based on a characterization of the launch system concept’s complexity 
and reliability. Inputs include launch vehicle system and subsystem design and 
technology descriptions, component/subsystem reliability information, and other cycle 
time impacts. The model estimates start-up (mostly facility development) and steady-state 
operations costs and includes information to characterize cycle time (or flight rate) 
requirements based on the number of bays provided in key spaceport facilities. This 
approach is summarized schematically in the “High-Level Overview of the Launch 
System Estimating Methodology” chart. 

The foundation for the estimates developed are Complexity vs. Reliability matrices 
developed for each of the 12 operations support functions. This approach characterizes 
the lowest cost system (from an operations perspective) as the least complex and most 
reliable design. During development of these matrices, select STS and Air Force aircraft 
data is used to determine staffing, cost, and cycle time ranges based on different 
complexity and reliability assumptions. 
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Figure 15, SOCM Planetary and Earth Orbiting Module Output Options 
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Figure 16, High-Level Overview of Launch System Estimating Methodology 

A sample complexity/reliability matrix for the Turnaround Facility/function is shown 
below. The values in the cells represent total headcount associated with a given 
complexity/reliability ranking. Separate matrices are used for labor, facility, GSE, and 
material/ODC costs for each of the 12 operations support functions. The complexity 
ranking compares the average time required per maintenance action and reliability 
captures the number of maintenance actions required. 
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Figure 17, Sample Complexity /Reliability Matrix 
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Because this approach provides only an operations-based perspective of a launch system 
concept’s performance, results need to be integrated with other system life cycle analysis 
tools to evaluate overall system performance, as shown in the figure below. These tools 
include development models to estimate non-recurring and recurring launch vehicle costs, 
market and economic models to characterize affordability, and simulation tools to 
integrate the results and aid end-to-end system design optimization. 



Figure 18, Integration of Models to Support Life Cycle Launch System Analyses 


Space Facilities Module . The Space Facilities Module is currently under development by 
JPL and will be integrated into SOCM by SAIC later in 1999. The design basis for the 
RPM effort is ISS and new data is being collected from the JSC Space Station Office. 
The current prototype includes 20 output cost categories and many “non-cost” 
performance outputs to characterize operational performance. Sample formats of this 
module’s outputs are provided on the following page. The ISS-based prototype is 
currently mid-way in development and should be complete by the middle of 1999. 
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Figure 19, Sample Output from the Space Facilities/Space Station Module 
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III. 1. 4. SOCM Study Products, Presentations, and Applications 

Available SOCM study products, presentations, and applications developed this contract 
year are listed in the following tables. 


Item 

Description 

SOCM Planetary and 
Earth Orbiting Rapid 
Prototype Models 
(2e, 2f, 2g) 

Rapid prototypes developed to incorporate user feedback, test 
estimation methodology options on various data sets, and as a 
communication aide between SOCM modeling teams, data 
collection teams, and the user community. SAIC is leading the 
development of these rapid prototypes and serving as a point- 
of-contact to collect and integrate information/data/ findings. 

SOCM User Manual 
and Guide (2g) 

A draft revised manual for version 2g is complete including an 
overall model methodology description. 

Launch System RPM 

Preliminary tool covering launch operations requirements for a 
turnaround facility 

Space Facilities RPM 

Updated version of the Model for Estimating Space Station 
Operations Costs (MESSOC) using current ISS program data 
and modem interface software. 


Figure 20, SOCM Study Products 


Item 

Description 

Quarterly Steering 

Committee Reports 
( J an/M ar/J un/ Aug) 

SAIC status updates for specific SOCM modules and 
presentation of advanced concepts; Includes data collection, 
performance assessment, and testing results for latest rapid 
prototype versions. 

Lessons Learned 

presentation to the 
ISE Program at 

NASA HQ (Apr) 

Provided a “Lessons-Leamed” briefing to the ISE cost 
community to use as a reference for developing advanced cost 
modeling concepts. 

NASA Cost 

Symposium at LaRC 
(Sep) 

Shared descriptions of SOCM modeling approaches and 
application experiences with the NASA cost community. 

Space Systems Cost 
Analysis Group 

Meeting at Motorola 
(Oct) 

Provided input to the SSCAG Operations and Support 
subgroup to facilitate better understanding of operational issues 
requiring attention 


Figure 21, SOCM Study Presentations 
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Item 

Description 

University Explorer 
(UNEX) AO 

Evaluation 

Used the Earth Orbiting Module to estimate costs for low-cost 
university missions ranging from less than $0.5M/yr to $2- 
3M/yr; Results were generally within 20% of proposed costs. 

Discovery Program 
AO Evaluation 

Over 30 proposals for planetary missions were evaluated with 
the Planetary Module; Results were within 10-20% of proposed 
costs for most mission candidates which varied substantially in 
annual staffing/cost requirements. 

Medium-Class 
Explorer (Midex) AO 
Evaluation 

Over 30 proposals for Earth orbiting space science missions 
were evaluated using the Earth Orbiting Module; Results were 
within 10-20% of proposed costs for most mission candidates 
which ranged from $lM/yr to over $10M/yr for MO&DA. 

Independent estimate 
of the GLAST 

mission for the SEU 
Office at GSFC 

Provided an estimate to the Structure and Evolution of the 
Universe Program Integration Office at GSFC; SOCM estimate 
was within 10% of a separate grass-roots estimate performed 
by the project team. 

JPL Project Design 
Center 

SOCM is available at the PDC and has been used by several 
Earth orbiting and planetary mission concept development 
teams to validate their grass-roots estimates. 

JPL Team X Venus 
Lander and Titan 
Aerobot/Lander 
Studies (Nov) 

Provided independent operations estimates using the Planetary 
Module to Team X in support of two advanced mission concept 
definition studies; Results were within 5% of the separate 
Team X estimates 


Figure 22, SOCM Study Applications 


III. 1. 5. Future Plans 

Future activities will focus on: 

- Completion of the Planetary and Earth Orbiting Module Version 1.0 with 
integrated help menus and documentation, 

- Completion of the Launch Systems Module incorporating MSFC, KSC, and 
LaRC inputs. 

Completion of the Space Facilities Module capturing the current ISS design with 
identified modifications to support lunar/Mars mission applications, 

- Begin integration of operations modeling tools with other models to support life 
cycle analyses 

- Development of initial RPMs to support HSF missions, and 
Continued support to the Steering Committee and SOCM user community. 
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III. 2. Air Force Cost Analysis Agency Taskings 
III.2. 1. NAFCOM Related Taskings 

There were several tasks funded by the Air Force Cost Analysis Agency that relate to the 
NAFCOM Cost Model. Progress made on each task is provided below. 

III. 2. 1 . 1 Complexity Generator 

The major efforts for the development of the Complexity Generators are described 
elsewhere in this final report. One specific action that the Air Force requested after 
reviewing the status of the Complexity Generators was to have someone with a strong 
statistical background review the development process. We have had two meetings with 
Dr. Steve Book of the Aerospace Corporation to get his input into our development of 
Complexity Generators. The latest meeting with Dr. Book included a briefing to Tony 
Fienfield and Lt. Col. Ray Carpio of the Air Force SMC/FMC. Dr. Book has suggested 
some new statistical approaches to the Complexity Generator development that are being 
incorporated. 

III. 2. 1. 2. PRICE Calibration 

We are calibrating the NAFCOM Air Force normalized data to the PRICE model. The 
approach we are using is the same as the one developed for the NASA normalized data. 
We have completed the ECIRP runs to derive the manufacturing complexity factors and 
the forward run of the model to adjust the inputs to match the design cost for all the 
subsystem level data. We are currently completing the calibration of the system 
integration level data. 

III. 2. 1.3. Software Cost Separation Study 

Currently the NAFCOM data base includes the cost of all software in the hardware cost. 
SAIC has completed a study on the separation of software cost from the avionics 
subsystems. In doing this study we determined which data points we had actual visibility 
into the software cost. Using these data points we developed an approach to normalize 
out the software cost for missions that did not have the required visibility. 

Our initial look at the backup cost data for NAFCOM indicates some thirty-five missions 
that provide software costs. These data include flight and ground support equipment 
(GSE) software costs. Software costs in the NAFCOM backup information is evident in 
some twenty-five unmanned missions, six planetary missions, two manned missions, and 
several launch vehicles. The division of these missions and vehicles is good given the fact 
that most of these data points occur in the 1990s time frame. Many of the unmanned 
missions are also of the “low-cost” variety. Listed below are some of the missions that 
have software cost visibility. 
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MISSION 

LAUNCH YEAR 

MISSION TYPE 

Lunar Prospector 

1999 

Planetary - Low Cost 

Mars Global Surveyor 

1998 

Planetary - Low Cost 

Step 3 

1998 

Earth Orbital - Low Cost 

Lewis 

1997 

Earth Orbital - Low Cost 

Mars Pathfinder 

1996 

Planetary - Low Cost 

TOMS-EP 

1996 

Earth Orbital - Low Cost 

NEAR 

1996 

Planetary - Low Cost 

HETE 

1995 

Earth Orbital - Low Cost 

STEP 2 

1994 

Earth Orbital - Low Cost 

STEP 1 

1994 

Earth Orbital - Low Cost 

STEPO 


Earth Orbital - Low Cost 

RADCAL 


Earth Orbital - Low Cost 

ALEXIS 

1993 

Earth Orbital - Low Cost 

MSTI 

1992 

Earth Orbital - Low Cost 

TOPEX 


Earth Orbital 

MICROSAT 

1991 

Earth Orbital - Low Cost 

UARS 

1991 

Earth Orbital 

GRO 

1991 

Earth Orbital 

HST 

1990 

Earth Orbital 

MACSAT 

1990 

Earth Orbital - Low Cost 


Figure 23, Missions with Software Cost Visibility 
III. 2. 1.4. SEER Calibration Study 

SAIC personnel met with persons from Galorath Incorporated to discuss the calibration of 
the NAFCOM data base to the SEER-H cost model. They expressed concerns over the 
lack of detailed cost and technical data at the sub-component levels. SEER-H requires 
input of the weight of electronics separate from the weight of structure within a 
subsystem. This requirement is similar to the input required for the PRICE model. We 
explained that our model is normally used in the early phases of a project and that low 
level input information is not available. They suggested an approach for the calibration 
that would not require the separation of circuit cards from the box. We are currently 
testing this approach with NAFCOM data and writing a report to document our planned 
approach to do this calibration. 
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III. 2. 1.5. Data Collection 


We have collected 13 new unmanned spacecraft data points for NAFCOM this year. In 
performing our normalization of the data, we have accounted for NASA ground rules and 
Air Force ground rules. 

III. 2. 1.6. NAFCOM Software Enhancements 

We completed several NAFCOM software enhancements this year. We developed a new 
summary and note area that makes better use of the NAFCOM estimating screen. This 
data is shown on the upper right portion of the display and provides an place to enter 
notes on your estimating assumptions for each CER or displays a summary of the cost 
estimate. 

We developed a new memo field in the project data base. This memo field includes a 
technical description for each of the subsystems in the data base. The memo field is 
accessed from the Specific Analogy screen within NAFCOM. As a user scrolls through 
the data points in the data base, the memo field displays technical data for the project that 
is selected. 

We also completed a metrics conversion feature within NAFCOM. When creating a CER 
the user will have a choice of entering data in either metric or English units. After 
entering the data in one type of units, it can be converted to the other units by selecting a 
button. 


III. 2. 2. AFCAA EELV Card Assessment 

SAIC, utilizing expertise in our Colorado Springs office, provide the Air Force Cost 
Analysis Agency with independent cost and technical analysis and evaluations relative to 
the government’s Acquisition and Operating and Support activities for the Evolved 
Expendable Launch Vehicle (EELV). Activities included comprehensive review of the 
EELV Cost Analysis Requirements Document (CARD) noting deficiencies, omissions, 
areas of potential misunderstanding, and errors. SAIC participate in Air Force reviews 
and reconciliation meetings. 

Specifically, SAIC participated in the Cost Integrated Product Team (CIPT) meeting in 
Washington on February 2 and March 11-12 to summarize technical review findings on 
the EELV CARD. We reviewed the two EELV prime contractors’ CARDs and prepared 
summaries of issues. SAIC reviewed on of the prime contractor’s Downselect Design 
Review documents to assess their risk mitigation efforts. SAIC also participated in ( 1 ) the 
CIPT Status Review in Los Angeles on February 10-11 to review program office cost 
estimates and (2) a meeting February 23 in Colorado Springs to review manpower and 
cost estimates. SAIC performed seven Special Interest Items independent evaluations and 
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one revised document review. Each evaluation was documented in a 2-3 page 
memorandum with specific technical review findings, complexity estimates, cost and 
editorial comments. SAIC also participated in the final reconciliation CIPT meeting in 
Los Angeles on April 16. 

III. 2. 3. AFCAA EELV Economic Assessment 

Cost and economic analysis support was provided to the AFCAA by participating on the 
Evolved Expendable Launch Vehicle (EELV) Investment/Financial Analysis (I/FA) 
Team. Support included participation in meetings to develop requirements and ground 
rules for a financial analysis model. NASA’s Reusable Launch Vehicle (RLV) economic 
analysis was presented for consideration due to similarities of the two programs (i.e. both 
proposed the commercial development and operation of a launch vehicle system). Some 
features of the RLV model were incorporated in the EELV analysis. An EELV financial 
analysis model was developed in Microsoft Excel. It was designed to calculate EELV 
launch prices based on vehicle engineering and manufacturing development (EMD) cost, 
production cost, operations cost, industry rate of return, national mission model, and 
investment payback period. Other features included cost spreading by fiscal year, 
production learning, and income taxes. The model was instrumental in developing the 
Service Cost Position and for estimating savings over the current expendable launch 
vehicle baseline. Sensitivity analyses were also conducted to determine cost sensitivity to 
the baseline set of model input assumptions. 


III. 3. COMPRE’ Model 

SAIC provided support to evaluate, review and validate the Complex Organizational 
Metric for Programmatic Risk Environments (COMPRE’) model. The work was 
performed for the MSFC System Analysis and Integration Laboratory between June and 
November of 1998. This effort was a follow-on task to an initial COMPRE’ model 
evaluation effort conducted in 1997. 

The COMPRE’ model, initially developed by MSFC and OR Applications, is an 
analytical program risk prediction model for aerospace hardware development programs. 
Utilized as a program management tool, COMPRE’ takes into account program 
characteristics including schedule and budget, technology and their respective maturity, 
relative budget investment, system architecture, and organizational considerations. When 
executed, COMPRE generates relative risk and technology payoff values as a function of 
the aforementioned parameters. Comparing the values allows the end user to gain insight 
into technology utilization, their associated risks and payoffs, and interactions with 
program management parameters, in systems approaching or in a development phase. 

The SAIC statement of work for this effort set forth the following tasks: 
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• Provide additional model validation and calibration (follow on to effort began in 
1997) 

• Conduct a pilot study of an existing MSFC project 

• Investigate and evaluate of an off-the-shelf software tool for suitability as a host 
platform and user interface 

• Provide data to OR Applications for continued theoretical development of the model 

III. 3. 1. Model Validation 


With respect to model validation and calibration, SAIC gathered historical cost, schedule 
and technology maturity levels from over thirty unmanned spacecraft programs. Once the 
data was placed into the COMPRE’ model, results for programmatic risk and potential 
technology payoff were tabulated and graphed. Examples of the tabular and graphical 
outputs from COMPRE’ are depicted below in Figures 1 and 2 respectively. 



Figure 19, COMPRE' Tabular Output Screen 



Figure 20, Program Risk vs. Technology Payoff Graph 
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When all of the analyzed programs were completed, composite charts depicting all of the Sigma (program 
risk) and Lambda (technology payoff) values were generated. This type of display is important to make a 
relative comparison of the programs. When graphed on a semi-log axis, the higher risk programs (i.e., 
higher Sigma value equates to higher program risk), with respect to each major milestone can be readily 
identified. This is shown in Figure 3. Likewise, the technology payoff chart. Figure 4, compares relative 
technology influences. 
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Figure 21, Comparison of Programmatic Risk 
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Figure 22, Comparison of Technology Payoff 
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III. 3. 2. Program Connectivity Diagram 

We were also responsible for generating program connectivity diagrams. An example of a 
diagram is depicted in Figure 5. The chart shows how systems, and subsystems, are 
interrelated. Generating the diagrams assisted us with understanding how programs were 
constructed and served as a visual aid for data entry purposes. The interrelationships are 
one of the four primary parameters COMPRE’ employs. 



III. 3. 3. Pilot Study 

After the model’s validation process was completed, we utilized the model in a pilot 
study program. Marshall’s Materials Science Research Facility (MSRF) agreed to provide 
programmatic data on one of their scientific packages. We held several meetings with the 
MSRF staff explaining the model functionality and data requirements. Once the data was 
acquired, it was introduced into the COMPRE’ model and analyzed. Due to the program’s 
relatively short duration (approximately two and a half years), COMPRE did not produce 
unusual risk outputs. In fact, the overall program risk was comparable to other programs 
of the same duration and cost magnitudes. It was noted however that even though the 
program required foreign (European Space Agency) participation, the overall 
contribution, from this particular management requirement, to program risk was not 
significant. 

III. 3. 4. Software Package Evaluation 

SAIC was also responsible for reviewing and evaluating a software package (MultiLinx) 
that could serve as a front end user interface or graphical adjunct for the COMPRE’ 
model. Multilinx is a unique connectivity visualization and analysis tool, which facilitates 
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the design and integration of complex electro-mechanical and software systems. The 
software program was evaluated because of its ability to access data files and draw the 
components specified by the user and their connecting paths. This would facilitate the 
drawing of interrelationships as depicted in Figure 5. 

After examining the program, we concluded that a considerable of amount of 
modification would be required to meet user (analyst’s) requirements. Either the software 
manufacturer would have to create new input screens or in-house programmers would be 
required to write interface code so it may be used as an adjunct graphical tool. Both 
options are costly and would have to be evaluated further in follow on tasks. 

III. 3. 5. OR Applications Interface 

As program data was populated throughout the model and outputs were produced, reports 
were periodically given to OR Applications for additional data synthesis. OR 
Applications used the COMPRE outputs to justify model basis functions, and to assist in 
the overall model validation process. In addition, data packages including mission 
descriptions, COMPRE tabular output (reference Figure 1), graphical output (reference 
Figure 2), interrelationship matrices, and summary level data were provided to OR 
Application and the government. Periodic meetings with OR Applications and the 
government were held to discuss the data packages, measure task progress, plan 
upcoming events, and resolve any outstanding issues. 

SAIC’s one person at the Ames Research Center in California has assisted several local 
NASA teams in developing mission proposals and project cost estimates over the last 
year. We have provided many detailed cost estimates using various models and 
techniques. The tools used include NAFCOM 96, REDSTAR, SICM, The Estimators 
Toolkit, and various custom Excel models. Both parametric and grassroots estimates were 
provided. Some of the missions supported in the Option Year 3 follow: 

III. 4. Microgravity Experiment Cost Model 

The Microgravity cost support was renewed in this contract year. Under this effort we 
were to re-program and update the Microgravity Experiment Cost Model. Other work 
included the potential collection of new microgravity cost data with assistance from 
personnel at MSFC. Data collection opportunities did not materialize during the year. 
We have completed the re-programming of the model. The model now operates in the 
Windows 97 mode and that is a major step forward from the previous version. New 
screens have been created which will further enhance the capability of the model. The 
model now operates on both Mac and PC computers. The new models are being 
distributed to MSFC Microgravity personnel. 


40 



III. 5. PRICE Calibration 


SAIC was tasked to continue to support NASA in calibrating the commercially available 
PRICE Cost Model for NASA users. Based on the response from the NASA community, 
this contract year we began calibrating the costs associated with integrating the 
subsystems into a system, or the Integration and Test (I&T) costs. This calibration is 
needed for an estimator to generate an estimate based totally on calibrated data. Since the 
PRICE calibrated data has been incorporated into NAFCOM at the subsystem level, it 
would be beneficial to have the calibrated integration and test values included as well. 

The first step was to develop a methodology for calibrating PRICE’S integration costs to 
NAFCOM ’s system level costs. SAIC worked closely with PRICE Systems personnel to 
develop a methodology that would correctly model and calibrate integration costs. The 
primary area of concern was how to model the System Test Hardware correctly. A cross 
section of data points were selected to be taken through the entire calibration process. 
This exercise allowed us to develop a methodology and PRICE factors for correctly 
calibrating System Level costs (or I&T costs). Adjustments were made as needed to the 
methodology and the selected data points were successfully calibrated. 

At this point, it was decided that SAIC would perform a re-calibration of all data points 
in order to stay consistent with NAFCOM. This re-calibration effort involved the 
following: 

Calibrating to “As Reported” data instead of the current “Synthetic Wraps 
Applied” data. 

Incorporating Air Force projects. 

Calculating new complexity values (NEWST and ECMPLX) based on the 
Complexity Generator’s TRL values. 

Calibrate integration values. 

New complexity values (NEWST and ECMPLX) were calculated based on the 
NAFCOM complexity generator analysis. It was felt that more information was known 
now due to this study. Therefore, we incorporated this new information and adjusted our 
complexity values. 

Currently we are in the process of calibrating to “As Reported” data. In order to do this 
process, spreadsheets were constructed which included the latest NAFCOM (NASA and 
Air Force) subsystem and system level cost data, complexity values, subsystem volumes, 
and development start dates. The platform values were reviewed and adjusted down 
slightly for the low-cost subsystems. 
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SAIC assisted PRICE in updating the model’s integration terminology to reflect current 
applications and meaningful integration processes. They plan to convert the current 
INTEGE/S tables to a generator. SAIC provided PRICE with some input on NASA's 
System Level Elements since this was an opportunity to have PRICE incorporate some 
NASA practices into the model. 

SAIC maintained contact with Del Wilson when possible through activity reports and 
phone conversations. We also assisted NASA PRICE Users as needed with PRICE 
questions. 

The series of PRICE Models was installed on two SAIC computers through the NASA 
newly obtained site license. This allows for PRICE to be on more computers and is 
accessible without the toggle box used before. 

III. 6. Ames Research Center Cost Analysis 

SAIC’s one person at the Ames Research Center in California has assisted several local 
NASA teams in developing mission proposals and project cost estimates over the last 
year. We have provided many detailed cost estimates using various models and 
techniques. The tools used include NAFCOM 96, REDSTAR, SICM, The Estimators 
Toolkit, and various custom Excel models. Both parametric and grassroots estimates were 
provided. Some of the missions supported in the Option Year 3 follow: 

III. 6.1. KEPLER Mission 

This is a spacebom photometer that will be used to find the sizes, orbits and frequencies 
of terrestrial and giant extrasolar planets. This is a DISCOVERY class mission 
responding to AO 98-OSS-04. The estimate, accompanying cost documents in the 
required format and explanatory text were delivered to the principal investigator in June 
and the mission is currently under evaluation by the DISCOVERY Program. 

III. 6. 2 VULCAN Camera 

This is a ground based extrasolar planet search being conducted by the Space Sciences 
Division at Ames. SAIC has been involved in this project on a number of levels. We have 
provided cost estimates for several projects including an observatory Dome Automation 
Project (completed in February, funded in March); and a full two-option Extended Search 
proposal to the ORIGINS Program (completed in May, competitively selected, and 
funded in November). We are currently working on estimates for a small Shutter 
Automation Project and a WAN connectivity plan for a radio link between Ames and the 
observatory on Mt Hamilton. 
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III. 6. 3. MAGE Mission 


The Mars Airborne Geophysical Explorer is a DISCOVERY class mission responding to 
AO 98-OSS-04. SAIC provided cost consulting and review services as well as assistance 
with the cost section of the proposal. We also provided costing feedback during the 
mission Red Team Review in May. The mission is currently under evaluation by the 
DISCOVERY Program. 

III. 6. 4. FISCAT Program 

The Far Infrared Survey Catalog is a Mission of Opportunity on the Japanese satellite 
ASTRO-F responding to AO-98-OSS-03. SAIC developed the cost estimate for the Ames 
portion of the mission and coordinated the cost assessment for the remaining U.S. 
segment of the mission. The estimates were culminated and the required documents were 
delivered in August. The mission is currently under evaluation by NASA Headquarters. 

III. 6. 5. SHARP-FX Mission Suite 

The Slender Hypervelocity Aerothermodynamic Research Probes are a series of reentry 
probe TPS technology demonstrators responding to the FUTURE-X Pathfinder Flight 
Demonstrators Program, NRA 8-22. For this mission, SAIC provided a complete end-to- 
end cost analysis of two reentry vehicles including the full set of the accompanying 
documents required by the NRA. These were finished and delivered in September. The 
mission is currently under evaluation by the FUTURE-X Program. 
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